Skip to content

[ntpl] Allow XRootD redirection with EOS files#20272

Open
vepadulano wants to merge 1 commit into
root-project:masterfrom
vepadulano:rntuple-eos-xrootd-redirection
Open

[ntpl] Allow XRootD redirection with EOS files#20272
vepadulano wants to merge 1 commit into
root-project:masterfrom
vepadulano:rntuple-eos-xrootd-redirection

Conversation

@vepadulano

@vepadulano vepadulano commented Nov 3, 2025

Copy link
Copy Markdown
Member

Fixes #20213

With these changes I can see the redirection happening when reading an RNTuple on EOS from lxplus (going from ntpl001_staff.root to root://eoshome-v.cern.ch//eos/user/v/vpadulan/ntpl001_staff.root

Breakpoint 1, ROOT::Internal::RRawFile::Create (url=..., options=...) at /tmp/vpadulan/root/io/io/src/RRawFile.cxx:88       
88         std::string transport = GetTransport(url);                                                                       
(gdb) n  
89         if (transport == "file") {
(gdb) p url                                                                                                                 
$1 = {_M_len = 18, _M_str = 0x4030a3 "ntpl001_staff.root"}  
[...]
103                    if (auto baseName = fileNameFromUrl.substr(fileNameFromUrl.find_last_of("/") + 1);
(gdb) 
105                       return Create(xurl, options);
(gdb) s
[...]
Breakpoint 1, ROOT::Internal::RRawFile::Create (url=..., options=...) at /tmp/vpadulan/root/io/io/src/RRawFile.cxx:88       
88         std::string transport = GetTransport(url);                                                                       
(gdb) n                                                                                                                     
89         if (transport == "file") {                                                                                       
(gdb) 
113        if (transport == "http" || transport == "https" ||
(gdb) p url
$8 = {_M_len = 64, _M_str = 0x136a790 "root://eoshome-v.cern.ch//eos/user/v/vpadulan/ntpl001_staff.root"}
(gdb) p transport
$9 = {static npos = 18446744073709551615, 
  _M_dataplus = {<std::allocator<char>> = {<std::__new_allocator<char>> = {<No data fields>}, <No data fields>}, 
    _M_p = 0x7fffffffa230 "root"}, _M_string_length = 4, {
    _M_local_buf = "root\000\177\000\000\220\242\377\377\377\177\000", _M_allocated_capacity = 139639930187634}}
(gdb)

There might be more places where the redirection needs to happen though.

@vepadulano vepadulano requested a review from jblomer November 3, 2025 09:54
@vepadulano vepadulano self-assigned this Nov 3, 2025
@vepadulano vepadulano requested a review from pcanal as a code owner November 3, 2025 09:54
jblomer
jblomer reviewed Nov 3, 2025
View reviewed changes

@jblomer jblomer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing this!

Comment thread io/io/src/RRawFile.cxx Outdated
Comment thread io/io/src/RRawFile.cxx Outdated
@github-actions

github-actions Bot commented Nov 3, 2025 •
edited
Loading

Copy link
Copy Markdown

Test Results

    21 files      21 suites   3d 3h 28m 48s ⏱️
 3 857 tests  3 815 ✅  0 💤  42 ❌
73 345 runs  73 175 ✅ 32 💤 138 ❌

For more details on these failures, see this check.

Results for commit d447fee.

♻️ This comment has been updated with latest results.

@vepadulano vepadulano force-pushed the rntuple-eos-xrootd-redirection branch from 3e7059b to ab97475 Compare November 3, 2025 22:36
@vepadulano vepadulano force-pushed the rntuple-eos-xrootd-redirection branch 7 times, most recently from 0ec1f2f to 4422f5e Compare November 4, 2025 07:46
pcanal
pcanal reviewed Nov 4, 2025
View reviewed changes
Comment thread io/io/src/IOUtils.cxx Outdated
pcanal
pcanal reviewed Nov 4, 2025
View reviewed changes
Comment thread io/io/src/IOUtils.cxx Outdated
@vepadulano vepadulano force-pushed the rntuple-eos-xrootd-redirection branch 2 times, most recently from c6f6051 to 45b2a23 Compare November 5, 2025 10:22
@vepadulano

vepadulano commented Nov 5, 2025

Copy link
Copy Markdown
Member Author

To the best of my understanding, this PR is simply moving the code that used to live separately in TFile::Open and RLoopManager::ChangeSpec extracting it into a common place. I have been testing these changes on EOS. Both TFile::Open and RNTupleReader::Open work flawlessly. For some reason, with the redirection happening in RDataFrame to store also the redirected URL as a sample ID, I see this error

terminate called after throwing an instance of 'std::runtime_error'
  what():  Full sample identifier 'root://eoshome-v.cern.ch//eos/user/v/vpadulan/rdf_sample.root/events' cannot be found in the available samples.

which is the same that triggered #17544 . What's even more worrying is that I don't see the error on every test execution, but sometimes the test also runs without failing. Before merging this PR I need to understand why that happens. It's possible the problem is still there somehow also without the code changes of this PR.

jblomer
jblomer reviewed Nov 5, 2025
View reviewed changes

@jblomer jblomer left a comment •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In principle looks good to me! We should think about how to test it. Extended attributes are file system dependent, so we may need to add some code just for testing, e.g. something that lets us control what GetXAttrVal returns under unit tests.

Comment thread io/io/inc/ROOT/InternalIOUtils.hxx Outdated
Comment thread io/io/src/InternalIOUtils.cxx Outdated
Comment thread io/io/src/InternalIOUtils.cxx Outdated
Comment thread io/io/src/InternalIOUtils.cxx Outdated
@vepadulano vepadulano closed this Nov 6, 2025
@vepadulano vepadulano reopened this Nov 6, 2025
@vepadulano vepadulano force-pushed the rntuple-eos-xrootd-redirection branch from 45b2a23 to 2794616 Compare November 6, 2025 18:24
@vepadulano

vepadulano commented Nov 10, 2025

Copy link
Copy Markdown
Member Author

Regarding the issues with the eos.url.xroot attribute being wrong:

I would wait for the latter issue to also be confirmed fix before moving on with this PR, FYI @amadio.

@jblomer

jblomer commented May 16, 2026

Copy link
Copy Markdown
Contributor

@vepadulano it looks like this PR can advance

amadio
amadio reviewed May 16, 2026
View reviewed changes
Comment thread io/io/src/InternalIOUtils.cxx Outdated
silverweed
silverweed reviewed May 18, 2026
View reviewed changes
Comment thread io/io/src/InternalIOUtils.cxx Outdated
Comment thread io/io/src/InternalIOUtils.cxx
@vepadulano
[io] Centralise XRootD EOS URL redirection
d447fee 
The logic to extract the extended attribute 'eos.url.xroot' from a path
to an EOS file on a FUSE mount is centralised in a separate header in
RIO, with an internal function called GetEOSRedirectedXRootUrl. This
function can be called anywhere needed in the rest of ROOT. With this
commit, the following places use this functionality:

- TFile::Open (done before this commit)
- RLoopManager::ChangeSpec (done before this commit)
- RRawFile::Create (introduced in this commit)

In particular the last means introducing the redirection also for
RNTupleReader, which follows a different logic to open the TFile.
@vepadulano vepadulano force-pushed the rntuple-eos-xrootd-redirection branch from 2794616 to d447fee Compare May 21, 2026 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@amadio amadio amadio left review comments
@jblomer jblomer jblomer left review comments
@pcanal pcanal pcanal left review comments
@silverweed silverweed silverweed left review comments
@bellenot bellenot Awaiting requested review from bellenot bellenot is a code owner
@martamaja10 martamaja10 Awaiting requested review from martamaja10
@hahnjo hahnjo Awaiting requested review from hahnjo
@enirolf enirolf Awaiting requested review from enirolf

At least 1 approving review is required to merge this pull request.

Assignees

@vepadulano vepadulano

Labels

in:RNTuple

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Implement RNTuple redirection from FUSE to XRootD

5 participants

@vepadulano @jblomer @amadio @pcanal @silverweed